Hugging Face's logo

conservationxlabs
/
miewid-msv2

Feature Extraction
Transformers
Safetensors
miewid
custom_code
Model card Files Community
lashao commited on
Commit
8c5bcdc
·
verified ·
1 Parent(s): a37ced9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +127 -163
README.md CHANGED
@@ -3,75 +3,67 @@ library_name: transformers
3
  tags: []
4
  ---
5
 
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
  ## Model Details
13
 
14
  ### Model Description
15
 
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
- This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
19
 
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
 
28
  ### Model Sources [optional]
29
 
30
  <!-- Provide the basic links for the model. -->
31
 
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
 
44
- [More Information Needed]
45
 
46
- ### Downstream Use [optional]
47
 
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
 
 
 
 
 
 
49
 
50
- [More Information Needed]
 
51
 
52
- ### Out-of-Scope Use
 
 
53
 
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
 
56
- [More Information Needed]
 
 
 
 
57
 
58
- ## Bias, Risks, and Limitations
 
59
 
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
61
 
62
- [More Information Needed]
 
63
 
64
- ### Recommendations
65
 
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
 
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
 
76
  ## Training Details
77
 
@@ -79,121 +71,93 @@ Use the code below to get started with the model.
79
 
80
  <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
 
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
 
199
- [More Information Needed]
 
3
  tags: []
4
  ---
5
 
 
 
 
 
 
 
6
  ## Model Details
7
 
8
  ### Model Description
9
 
10
+ MiewID-msv2 is a feature extractor pretrained re-identification task on a large, high-quality dataset of 54 wildlife species - terrestrial and aquatic - including fins, flukes, flanks, faces.
 
 
11
 
12
+ - **Model Type:** Wildlife re-identification feature backbone
13
+ - **Model Stats:**
14
+ - Params (M): 51.11
15
+ - GMACs: 24.38
16
+ - Activations (M): 91.11
17
+ - Image size: 440 x 440
 
18
 
19
  ### Model Sources [optional]
20
 
21
  <!-- Provide the basic links for the model. -->
22
 
23
+ - **Repository:** https://github.com/WildMeOrg/wbia-plugin-miew-id
24
+ - **Backbone:** https://huggingface.co/timm/efficientnetv2_rw_m.agc_in1k
 
 
 
 
 
 
 
 
 
25
 
26
+ ## Usage
27
 
28
+ Intended use is re-identification of individuals from different species by matching against a database of grount-truth samples. Model features can also be used for species classification by retrieval.
29
 
30
+ ### Embedding Extraction
31
+ ```
32
+ import numpy as np
33
+ from PIL import Image
34
+ import torch
35
+ import torchvision.transforms as transforms
36
+ from transformers import AutoModel
37
 
38
+ model_tag = f"lashao/miewid-msv2"
39
+ model = AutoModel.from_pretrained(model_tag, trust_remote_code=True)
40
 
41
+ def generate_random_image(height=440, width=440, channels=3):
42
+ random_image = np.random.randint(0, 256, (height, width, channels), dtype=np.uint8)
43
+ return Image.fromarray(random_image)
44
 
45
+ random_image = generate_random_image()
46
 
47
+ preprocess = transforms.Compose([
48
+ transforms.Resize((440, 440)),
49
+ transforms.ToTensor(),
50
+ transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
51
+ ])
52
 
53
+ input_tensor = preprocess(random_image)
54
+ input_batch = input_tensor.unsqueeze(0)
55
 
56
+ with torch.no_grad():
57
+ output = model(input_batch)
58
 
59
+ print(output)
60
+ print(output.shape )
61
 
62
+ ```
63
 
64
+ ### More Examples
65
 
66
+ View more usage examples at https://github.com/WildMeOrg/wbia-plugin-miew-id/tree/main/wbia_miew_id/datasets
 
 
 
 
 
 
67
 
68
  ## Training Details
69
 
 
71
 
72
  <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
73
 
74
+ The dataset used for these experiments was a combination of data from Wildbook platforms (multiple users) and
75
+ the Happywhale Kaggle competitions multi-species dataset. The latter is available for
76
+ non-commercial purposes and academic research and education. A subset of data from Wildbook platforms is available at https://lila.science/datasets. There are two main distinctions
77
+ between the Happywhale and Wildbook datasets in addition to different species coverage as used
78
+ in our experiments.
79
+ The Happywhale dataset did not have viewpoint labels (e.g., left, right, top, bottom), which can
80
+ refine re-ID training approaches.
81
+ The bounding boxes for the annotations of interest (AoI) in the Happywhale-sourced data were
82
+ generated using a pre-trained Detic model (developed during the contest).
83
+ Where the datasets overlapped by species, only one dataset was used to ensure no duplication of
84
+ individuals was present. Not every species from the Happywhale dataset was incorporated, with
85
+ limited data excluding some underrepresented species wholesale. For selected species, no
86
+ filtering was done on the basis of quality or distinctiveness, with real-world data valued for its
87
+ diversity.
88
+
89
+ ### Example images
90
+
91
+
92
+ ![image/jpg](https://cthulhu.dyn.wildme.io/public/datasets/msv2_dataset_example.jpg)
93
+
94
+ ## Evaluation Results
95
+
96
+ The multi-species model shows improvements over models trained on single species. Moreover the model shows strong generalization ability for majority of the species when trained and evaluated in a leave-one-out fashion.
97
+
98
+ ### Detailed results
99
+
100
+ |Source |group |map |rank-1|rank-5|rank-10|rank-20|viewpoints |n_test_annots|n_test_names|n_train_annots|n_train_names|
101
+ |--------------|--------------------------------|-----|------|------|-------|-------|--------------------------------------------------------------------------------------------------------------------|-------------|------------|--------------|-------------|
102
+ |Wildbook |amur_tiger |91.8 |100.0 |100.0 |100.0 |100.0 |['left', 'right'] |233 |47 |691 |75 |
103
+ |Wildbook |beluga_whale |61.36|71.85 |81.63 |85.51 |89.28 |['up'] |849 |228 |2767 |354 |
104
+ |Happywhale |blue_whale |36.98|38.68 |55.11 |61.52 |69.04 |['unknown'] |998 |392 |1185 |339 |
105
+ |Wildbook |bottlenose_dolphin |85.69|93.99 |96.18 |96.85 |97.52 |['right', 'left'] |4911 |838 |11607 |1062 |
106
+ |Happywhale |brydes_whale |69.96|80.95 |92.86 |95.24 |100.0 |['unknown'] |42 |9 |81 |15 |
107
+ |Lomas Capuchin|capuchin |34.8 |47.97 |68.58 |76.69 |85.14 |['front'] |296 |35 |1708 |29 |
108
+ |Wildbook |cheetah |57.37|71.15 |84.92 |87.87 |92.13 |['left', 'right'] |624 |139 |1522 |144 |
109
+ |Wildbook |chimpanzee |94.7 |100.0 |100.0 |100.0 |100.0 |['unknown'] |52 |8 |138 |9 |
110
+ |C-Tai |chimpanzee_ctai |51.67|69.64 |83.68 |89.18 |94.12 |['front'] |527 |63 |2680 |44 |
111
+ |C-Zoo |chimpanzee_czoo |77.66|86.67 |93.33 |97.5 |99.17 |['front'] |240 |24 |1258 |17 |
112
+ |Happywhale |commersons_dolphin |0.0 |0.0 |0.0 |0.0 |0.0 |['unknown'] |10 |5 |7 |2 |
113
+ |Happywhale |cuviers_beaked_whale |52.89|55.71 |75.71 |82.86 |90.0 |['unknown'] |70 |28 |89 |21 |
114
+ |DogFaceNet |dog |82.96|87.38 |94.78 |96.74 |97.99 |['unknown'] |1838 |544 |5750 |1088 |
115
+ |Happywhale |dusky_dolphin |85.82|83.54 |93.04 |94.3 |94.94 |['unknown'] |158 |68 |154 |41 |
116
+ |Wildbook |eurasianlynx |58.8 |67.82 |83.78 |88.83 |92.02 |['left', 'right'] |726 |199 |2251 |285 |
117
+ |Happywhale |fin_whale |67.65|76.92 |87.18 |90.11 |93.04 |['unknown'] |273 |71 |533 |55 |
118
+ |Happywhale |frasiers_dolphin |0.0 |0.0 |0.0 |0.0 |0.0 | |-1 |-1 |-1 |-1 |
119
+ |Wildbook |giraffe |99.18|99.08 |99.08 |100.0 |100.0 |['right', 'front', 'left', 'back'] |423 |162 |2540 |426 |
120
+ |Wildbook |horse_wild+face |0.0 |0.0 |0.0 |0.0 |0.0 |['front'] |17 |7 |174 |53 |
121
+ |Wildbook |horse_wild_tunisian+face |79.59|98.53 |99.51 |100.0 |100.0 |['unknown'] |204 |39 |999 |37 |
122
+ |Wildbook |hyena |64.17|79.32 |89.22 |93.14 |96.41 |['right', 'left'] |1231 |301 |1154 |201 |
123
+ |Wildbook |hyperoodon_ampullatus |91.14|96.11 |98.12 |98.34 |98.84 |['left', 'right'] |1930 |354 |24952 |759 |
124
+ |Wildbook |jaguar |67.45|81.72 |91.04 |92.91 |94.4 |['left', 'right'] |540 |124 |980 |103 |
125
+ |PrimFace |japanese_monkey |89.05|93.85 |98.46 |98.46 |100.0 |['unknown'] |65 |12 |219 |14 |
126
+ |Happywhale |killer_whale |62.82|77.13 |85.48 |88.81 |91.82 |['unknown'] |599 |145 |1379 |177 |
127
+ |Wildbook |leopard |65.03|82.64 |90.23 |92.83 |94.46 |['right', 'left'] |595 |108 |4261 |381 |
128
+ |Wildbook |lion |87.12|93.86 |100.0 |100.0 |100.0 |['front', 'unknown', 'frontright', 'frontleft', 'upfront', 'right', 'upleft', 'left', 'upright', 'backleft', 'back']|1364 |242 |3981 |241 |
129
+ |Happywhale |long_finned_pilot_whale |99.06|98.11 |100.0 |100.0 |100.0 |['unknown'] |53 |21 |33 |10 |
130
+ |Wildbook |lynx_pardinus |48.22|57.68 |73.44 |77.18 |84.96 |['left', 'right'] |467 |142 |1376 |259 |
131
+ |MacaqueFaces |macaque_face |89.87|95.88 |97.94 |99.12 |100.0 |['front'] |340 |34 |3480 |25 |
132
+ |Happywhale |melon_headed_whale |90.8 |93.42 |96.05 |96.71 |98.68 |['unknown'] |152 |55 |180 |43 |
133
+ |Wildbook |nyala |64.64|70.65 |90.05 |94.03 |97.01 |['left', 'right'] |418 |99 |1303 |149 |
134
+ |Happywhale |pantropic_spotted_dolphin |78.15|85.19 |90.74 |94.44 |96.3 |['unknown'] |54 |11 |55 |8 |
135
+ |Happywhale |pygmy_killer_whale |89.07|84.62 |100.0 |100.0 |100.0 |['unknown'] |26 |10 |32 |5 |
136
+ |PrimFace |rhesus_monkey |78.12|89.13 |97.28 |98.91 |100.0 |['unknown'] |184 |28 |478 |24 |
137
+ |Happywhale |rough_toothed_dolphin |0.0 |0.0 |0.0 |0.0 |0.0 |['unknown'] |6 |3 |7 |2 |
138
+ |SealID |seal |37.93|59.51 |80.0 |86.1 |91.71 |['unknown'] |410 |41 |1448 |36 |
139
+ |Happywhale |sei_whale |67.86|71.55 |87.07 |93.1 |95.69 |['unknown'] |116 |35 |106 |29 |
140
+ |Wildbook |short_fin_pilot_whale+fin_dorsal|92.49|92.31 |98.08 |100.0 |100.0 |['right', 'left'] |272 |64 |1290 |137 |
141
+ |Happywhale |short_finned_pilot_whale |94.39|94.96 |95.8 |95.8 |97.48 |['unknown'] |119 |57 |67 |17 |
142
+ |Wildbook |snow_leopard |62.43|82.45 |88.91 |92.15 |94.53 |['left', 'right'] |927 |144 |2307 |99 |
143
+ |Happywhale |spinner_dolphin |99.88|100.0 |100.0 |100.0 |100.0 |['left', 'right'] |335 |97 |1562 |251 |
144
+ |Happywhale |spotted_dolphin |84.92|83.67 |89.8 |95.92 |97.96 |['unknown'] |49 |22 |128 |28 |
145
+ |Wildbook |turtle_green |100.0|100.0 |100.0 |100.0 |100.0 |['left', 'right', 'up', 'front'] |543 |109 |5097 |76 |
146
+ |Wildbook |turtle_green+head |95.1 |96.02 |100.0 |100.0 |100.0 |['left', 'right', 'up', 'front'] |544 |108 |5737 |76 |
147
+ |Wildbook |turtle_hawksbill |78.84|81.15 |88.37 |92.56 |95.81 |['up', 'left', 'right', 'front'] |506 |106 |5397 |71 |
148
+ |Wildbook |turtle_hawksbill+head |86.78|91.74 |95.45 |96.69 |98.35 |['right', 'left', 'up', 'front'] |589 |109 |7033 |71 |
149
+ |SeaTurtleID |turtle_loggerhead_ext |63.67|75.43 |86.61 |90.17 |92.87 |['unknown', 'left', 'top', 'right'] |815 |217 |6573 |355 |
150
+ |SeaTurtleID |turtle_loggerhead_ext+head |94.14|94.63 |100.0 |100.0 |100.0 |['left', 'right', 'topright', 'top', 'topleft', 'below', 'front'] |778 |209 |6361 |348 |
151
+ |Wildbook |whale_fin+fin_dorsal |82.12|85.12 |93.87 |95.04 |95.75 |['left', 'right'] |394 |110 |1629 |186 |
152
+ |Wildbook |whale_grey |90.97|91.55 |95.54 |96.18 |96.82 |['right', 'left'] |381 |103 |5662 |758 |
153
+ |Wildbook |whale_humpback+fin_dorsal |80.3 |84.07 |90.08 |91.6 |95.42 |['right', 'left'] |509 |194 |3734 |541 |
154
+ |Wildbook |whale_humpback+fluke |97.11|96.67 |100.0 |100.0 |100.0 |['back'] |150 |64 |1151 |356 |
155
+ |Wildbook |whale_orca |97.01|98.36 |99.18 |100.0 |100.0 |['right', 'left'] |249 |49 |2790 |435 |
156
+ |Wildbook |whale_orca+fin_dorsal |95.41|95.87 |97.52 |97.52 |99.17 |['right', 'left'] |623 |187 |3978 |430 |
157
+ |Wildbook |whale_sperm+fluke |96.14|97.87 |98.33 |98.48 |99.09 |['back'] |658 |125 |25849 |558 |
158
+ |Wildbook |whaleshark |57.61|72.25 |79.96 |82.27 |87.21 |['left', 'right'] |997 |212 |5713 |376 |
159
+ |Wildbook |white_shark+fin_dorsal |91.4 |94.89 |97.87 |98.54 |99.27 |['left', 'right'] |322 |88 |1067 |162 |
160
+ |Happywhale |white_sided_dolphin |81.82|76.0 |88.0 |88.0 |100.0 |['unknown'] |25 |11 |13 |4 |
161
+ |Wildbook |wilddog |73.4 |85.11 |90.93 |92.38 |94.07 |['left', 'right'] |3460 |848 |4217 |656 |
162
+ |Wildbook |zebra_grevys |91.79|96.79 |97.57 |97.74 |98.04 |['right'] |1684 |332 |8927 |362 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
163