File size: 19,138 Bytes
a37ced9 a9077b6 a37ced9 65555b5 a37ced9 8c5bcdc a37ced9 8c5bcdc a37ced9 a9077b6 a37ced9 8c5bcdc a37ced9 8c5bcdc a37ced9 85cc80e 8c5bcdc a37ced9 8c5bcdc a37ced9 8c5bcdc a37ced9 8c5bcdc a37ced9 8c5bcdc a37ced9 8c5bcdc a37ced9 8c5bcdc a942142 a37ced9 8c5bcdc a37ced9 8c5bcdc a37ced9 74201a3 a37ced9 a9077b6 a37ced9 dda4e58 65555b5 8c5bcdc a9077b6 8c5bcdc a37ced9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 |
---
library_name: transformers
tags: []
---
# Model Details
### Model Description
MiewID-msv2 is a feature extractor trained for re-identification using contrastive learning on a large, high-quality dataset of 54 wildlife species - terrestrial and aquatic - including fins, flukes, flanks, faces.
- **Model Type:** Wildlife re-identification feature backbone
- **Model Stats:**
- Params (M): 51.11
- GMACs: 24.38
- Activations (M): 91.11
- Image size: 440 x 440
### Model Sources [optional]
<!-- Provide the basic links for the model. -->
- **Repository:** https://github.com/WildMeOrg/wbia-plugin-miew-id
- **Backbone:** https://huggingface.co/timm/efficientnetv2_rw_m.agc_in1k
# Usage
Intended use is re-identification of individuals from different species by matching against a database of grount-truth samples. Model features can also be used for species classification by retrieval.
### Embedding Extraction
```
import numpy as np
from PIL import Image
import torch
import torchvision.transforms as transforms
from transformers import AutoModel
model_tag = f"conservationxlabs/miewid-msv2"
model = AutoModel.from_pretrained(model_tag, trust_remote_code=True)
def generate_random_image(height=440, width=440, channels=3):
random_image = np.random.randint(0, 256, (height, width, channels), dtype=np.uint8)
return Image.fromarray(random_image)
random_image = generate_random_image()
preprocess = transforms.Compose([
transforms.Resize((440, 440)),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
input_tensor = preprocess(random_image)
input_batch = input_tensor.unsqueeze(0)
with torch.no_grad():
output = model(input_batch)
print(output)
print(output.shape)
```
### More Examples
View more usage examples at https://github.com/WildMeOrg/wbia-plugin-miew-id/tree/main/wbia_miew_id/examples
# Training Details
### Training Data
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
The dataset used for these experiments was a combination of data from Wildbook platforms (multiple users), Happywhale Kaggle competitions multi-species dataset and multiple publicly available datasets. A small subset of data from Wildbook platforms is available at https://lila.science/datasets.
### Example images
# Evaluation Results
The multi-species model shows improvements over models trained on single species. Moreover the model shows strong generalization ability for majority of the species when trained and evaluated in a leave-one-out fashion.
### Detailed results
|Source |group |map |rank-1|rank-5|rank-10|rank-20|viewpoints |n_test_annots|n_test_names|n_train_annots|n_train_names|
|--------------|--------------------------------|-----|------|------|-------|-------|--------------------------------------------------------------------------------------------------------------------|-------------|------------|--------------|-------------|
|Wildbook |amur_tiger |91.8 |100.0 |100.0 |100.0 |100.0 |['left', 'right'] |233 |47 |691 |75 |
|Wildbook |beluga_whale |61.36|71.85 |81.63 |85.51 |89.28 |['up'] |849 |228 |2767 |354 |
|Happywhale |blue_whale |36.98|38.68 |55.11 |61.52 |69.04 |['unknown'] |998 |392 |1185 |339 |
|Wildbook |bottlenose_dolphin |85.69|93.99 |96.18 |96.85 |97.52 |['right', 'left'] |4911 |838 |11607 |1062 |
|Happywhale |brydes_whale |69.96|80.95 |92.86 |95.24 |100.0 |['unknown'] |42 |9 |81 |15 |
|Lomas Capuchin|capuchin |34.8 |47.97 |68.58 |76.69 |85.14 |['front'] |296 |35 |1708 |29 |
|Wildbook |cheetah |57.37|71.15 |84.92 |87.87 |92.13 |['left', 'right'] |624 |139 |1522 |144 |
|Wildbook |chimpanzee |94.7 |100.0 |100.0 |100.0 |100.0 |['unknown'] |52 |8 |138 |9 |
|C-Tai |chimpanzee_ctai |51.67|69.64 |83.68 |89.18 |94.12 |['front'] |527 |63 |2680 |44 |
|C-Zoo |chimpanzee_czoo |77.66|86.67 |93.33 |97.5 |99.17 |['front'] |240 |24 |1258 |17 |
|Happywhale |commersons_dolphin |0.0 |0.0 |0.0 |0.0 |0.0 |['unknown'] |10 |5 |7 |2 |
|Happywhale |cuviers_beaked_whale |52.89|55.71 |75.71 |82.86 |90.0 |['unknown'] |70 |28 |89 |21 |
|DogFaceNet |dog |82.96|87.38 |94.78 |96.74 |97.99 |['unknown'] |1838 |544 |5750 |1088 |
|Happywhale |dusky_dolphin |85.82|83.54 |93.04 |94.3 |94.94 |['unknown'] |158 |68 |154 |41 |
|Wildbook |eurasianlynx |58.8 |67.82 |83.78 |88.83 |92.02 |['left', 'right'] |726 |199 |2251 |285 |
|Happywhale |fin_whale |67.65|76.92 |87.18 |90.11 |93.04 |['unknown'] |273 |71 |533 |55 |
|Happywhale |frasiers_dolphin |0.0 |0.0 |0.0 |0.0 |0.0 | |-1 |-1 |-1 |-1 |
|Wildbook |giraffe |99.18|99.08 |99.08 |100.0 |100.0 |['right', 'front', 'left', 'back'] |423 |162 |2540 |426 |
|Wildbook |horse_wild+face |0.0 |0.0 |0.0 |0.0 |0.0 |['front'] |17 |7 |174 |53 |
|Wildbook |horse_wild_tunisian+face |79.59|98.53 |99.51 |100.0 |100.0 |['unknown'] |204 |39 |999 |37 |
|Wildbook |hyena |64.17|79.32 |89.22 |93.14 |96.41 |['right', 'left'] |1231 |301 |1154 |201 |
|Wildbook |hyperoodon_ampullatus |91.14|96.11 |98.12 |98.34 |98.84 |['left', 'right'] |1930 |354 |24952 |759 |
|Wildbook |jaguar |67.45|81.72 |91.04 |92.91 |94.4 |['left', 'right'] |540 |124 |980 |103 |
|PrimFace |japanese_monkey |89.05|93.85 |98.46 |98.46 |100.0 |['unknown'] |65 |12 |219 |14 |
|Happywhale |killer_whale |62.82|77.13 |85.48 |88.81 |91.82 |['unknown'] |599 |145 |1379 |177 |
|Wildbook |leopard |65.03|82.64 |90.23 |92.83 |94.46 |['right', 'left'] |595 |108 |4261 |381 |
|Wildbook |lion |87.12|93.86 |100.0 |100.0 |100.0 |['front', 'unknown', 'frontright', 'frontleft', 'upfront', 'right', 'upleft', 'left', 'upright', 'backleft', 'back']|1364 |242 |3981 |241 |
|Happywhale |long_finned_pilot_whale |99.06|98.11 |100.0 |100.0 |100.0 |['unknown'] |53 |21 |33 |10 |
|Wildbook |lynx_pardinus |48.22|57.68 |73.44 |77.18 |84.96 |['left', 'right'] |467 |142 |1376 |259 |
|MacaqueFaces |macaque_face |89.87|95.88 |97.94 |99.12 |100.0 |['front'] |340 |34 |3480 |25 |
|Happywhale |melon_headed_whale |90.8 |93.42 |96.05 |96.71 |98.68 |['unknown'] |152 |55 |180 |43 |
|Wildbook |nyala |64.64|70.65 |90.05 |94.03 |97.01 |['left', 'right'] |418 |99 |1303 |149 |
|Happywhale |pantropic_spotted_dolphin |78.15|85.19 |90.74 |94.44 |96.3 |['unknown'] |54 |11 |55 |8 |
|Happywhale |pygmy_killer_whale |89.07|84.62 |100.0 |100.0 |100.0 |['unknown'] |26 |10 |32 |5 |
|PrimFace |rhesus_monkey |78.12|89.13 |97.28 |98.91 |100.0 |['unknown'] |184 |28 |478 |24 |
|Happywhale |rough_toothed_dolphin |0.0 |0.0 |0.0 |0.0 |0.0 |['unknown'] |6 |3 |7 |2 |
|SealID |seal |37.93|59.51 |80.0 |86.1 |91.71 |['unknown'] |410 |41 |1448 |36 |
|Happywhale |sei_whale |67.86|71.55 |87.07 |93.1 |95.69 |['unknown'] |116 |35 |106 |29 |
|Wildbook |short_fin_pilot_whale+fin_dorsal|92.49|92.31 |98.08 |100.0 |100.0 |['right', 'left'] |272 |64 |1290 |137 |
|Happywhale |short_finned_pilot_whale |94.39|94.96 |95.8 |95.8 |97.48 |['unknown'] |119 |57 |67 |17 |
|Wildbook |snow_leopard |62.43|82.45 |88.91 |92.15 |94.53 |['left', 'right'] |927 |144 |2307 |99 |
|Happywhale |spinner_dolphin |99.88|100.0 |100.0 |100.0 |100.0 |['left', 'right'] |335 |97 |1562 |251 |
|Happywhale |spotted_dolphin |84.92|83.67 |89.8 |95.92 |97.96 |['unknown'] |49 |22 |128 |28 |
|Wildbook |turtle_green |100.0|100.0 |100.0 |100.0 |100.0 |['left', 'right', 'up', 'front'] |543 |109 |5097 |76 |
|Wildbook |turtle_green+head |95.1 |96.02 |100.0 |100.0 |100.0 |['left', 'right', 'up', 'front'] |544 |108 |5737 |76 |
|Wildbook |turtle_hawksbill |78.84|81.15 |88.37 |92.56 |95.81 |['up', 'left', 'right', 'front'] |506 |106 |5397 |71 |
|Wildbook |turtle_hawksbill+head |86.78|91.74 |95.45 |96.69 |98.35 |['right', 'left', 'up', 'front'] |589 |109 |7033 |71 |
|SeaTurtleID |turtle_loggerhead_ext |63.67|75.43 |86.61 |90.17 |92.87 |['unknown', 'left', 'top', 'right'] |815 |217 |6573 |355 |
|SeaTurtleID |turtle_loggerhead_ext+head |94.14|94.63 |100.0 |100.0 |100.0 |['left', 'right', 'topright', 'top', 'topleft', 'below', 'front'] |778 |209 |6361 |348 |
|Wildbook |whale_fin+fin_dorsal |82.12|85.12 |93.87 |95.04 |95.75 |['left', 'right'] |394 |110 |1629 |186 |
|Wildbook |whale_grey |90.97|91.55 |95.54 |96.18 |96.82 |['right', 'left'] |381 |103 |5662 |758 |
|Wildbook |whale_humpback+fin_dorsal |80.3 |84.07 |90.08 |91.6 |95.42 |['right', 'left'] |509 |194 |3734 |541 |
|Wildbook |whale_humpback+fluke |97.11|96.67 |100.0 |100.0 |100.0 |['back'] |150 |64 |1151 |356 |
|Wildbook |whale_orca |97.01|98.36 |99.18 |100.0 |100.0 |['right', 'left'] |249 |49 |2790 |435 |
|Wildbook |whale_orca+fin_dorsal |95.41|95.87 |97.52 |97.52 |99.17 |['right', 'left'] |623 |187 |3978 |430 |
|Wildbook |whale_sperm+fluke |96.14|97.87 |98.33 |98.48 |99.09 |['back'] |658 |125 |25849 |558 |
|Wildbook |whaleshark |57.61|72.25 |79.96 |82.27 |87.21 |['left', 'right'] |997 |212 |5713 |376 |
|Wildbook |white_shark+fin_dorsal |91.4 |94.89 |97.87 |98.54 |99.27 |['left', 'right'] |322 |88 |1067 |162 |
|Happywhale |white_sided_dolphin |81.82|76.0 |88.0 |88.0 |100.0 |['unknown'] |25 |11 |13 |4 |
|Wildbook |wilddog |73.4 |85.11 |90.93 |92.38 |94.07 |['left', 'right'] |3460 |848 |4217 |656 |
|Wildbook |zebra_grevys |91.79|96.79 |97.57 |97.74 |98.04 |['right'] |1684 |332 |8927 |362 |
|