ibm-granite
/

granite-vision-3.3-2b-embedding

@@ -17,23 +17,23 @@ We evaluated granite-vision-3.3-2b-embedding alongside other top colBERT style m
 ## **NDCG@5 - ViDoRe V2**
 | Collection \ Model                     | ColPali-v1.3 | ColQwen2.5-v0.2 | ColNomic-3b |  ColSmolvlm-v0.1     |  granite-vision-3.3-2b-embedding |
 |----------------------------------------|--------------|------------------|-------------|-------------------|-----------
-| ESG Restaurant Human                   | 51.10        | 68.40            | 65.80       |    62.4               | 62.30                    |
-| Economics Macro Multilingual           | 49.90        | 56.50            | 55.40       |     47.4              | 48.30                    |
-| MIT Biomedical                         | 59.70        | 63.60            | 63.50       |    58.1               |60.00                    |
-| ESG Restaurant Synthetic               | 57.00        | 57.40            | 56.60       |     51.1              |54.00                    |
-| ESG Restaurant Synthetic Multilingual  | 55.70        | 57.40            | 57.20       |     47.6             |53.50                    |
-| MIT Biomedical Multilingual            | 56.50        | 61.10            | 62.50       |      50.5             | 53.60                    |
-| Economics Macro                        | 51.60        | 59.80            | 60.20       |      60.9            |60.00                    |
-| **Avg (ViDoRe2)**                      | **54.50**    | **60.60**        | **60.17**   | **54**.              |**55.96**                    |
 ## **NDCG@5 - REAL-MM-RAG**
 | Collection \ Model                     | ColPali-v1.3 | ColQwen2.5-v0.2 | ColNomic-3b |   ColSmolvlm-v0.1            |  granite-vision-3.3-2b-embedding |
 |----------------------------------------|--------------|------------------|-------------|--------------------------| ------------------
-| FinReport                              | 0.55         | 0.66             | 0.78        |   0.65                  |0.70
-| FinSlides                              | 0.68         | 0.79             | 0.81        |   0.55                 |0.74
-| TechReport                             | 0.78         | 0.86             | 0.88        |   0.83                 |0.84
-| TechSlides                             | 0.90         | 0.93             | 0.92        |   0.91            |0.93
-| **Avg (REAL-MM-RAG)**                  | **0.73**     | **0.81**         | **0.85**    |   **0.74**           |**0.80**
 - **Release Date**: June 11th 2025
 - **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
@@ -105,7 +105,13 @@ print("=" * 50)
 For an example of MM-RAG using granite-vision-3.3-2b-embedding refer to [this notebook](......).
 **Model Architecture:**
-To Be Updated
 **Training Data:**
 Our training data is entirly comprised from DocFM. DocFM is a large-scale comprehensive dataset effort at IBM consisting of 85 million document pages extracted from unique PDF

 ## **NDCG@5 - ViDoRe V2**
 | Collection \ Model                     | ColPali-v1.3 | ColQwen2.5-v0.2 | ColNomic-3b |  ColSmolvlm-v0.1     |  granite-vision-3.3-2b-embedding |
 |----------------------------------------|--------------|------------------|-------------|-------------------|-----------
+| ESG Restaurant Human                   | 51.1        | 68.4           | 65.8       |    62.4               | 62.3                    |
+| Economics Macro Multilingual           | 49.9        | 56.5            | 55.4       |     47.4              | 48.3                    |
+| MIT Biomedical                         | 59.7        | 63.6            | 63.5       |    58.1               |60.0                   |
+| ESG Restaurant Synthetic               | 57.0        | 57.4            | 56.6       |     51.1              |54.0                    |
+| ESG Restaurant Synthetic Multilingual  | 55.7        | 57.4            | 57.2       |     47.6             |53.5                    |
+| MIT Biomedical Multilingual            | 56.5        | 61.1            | 62.5       |      50.5             | 53.6                    |
+| Economics Macro                        | 51.6        | 59.8            | 60.2       |      60.9            |60.0                    |
+| **Avg (ViDoRe2)**                      | **54.5**    | **60.6**        | **60.2**   | **54.0**.              |**56.0**                    |
 ## **NDCG@5 - REAL-MM-RAG**
 | Collection \ Model                     | ColPali-v1.3 | ColQwen2.5-v0.2 | ColNomic-3b |   ColSmolvlm-v0.1            |  granite-vision-3.3-2b-embedding |
 |----------------------------------------|--------------|------------------|-------------|--------------------------| ------------------
+| FinReport                              | 55         | 66             | 78        |   65                  |70
+| FinSlides                              | 68        | 79             | 81        |   55                 |74
+| TechReport                             | 78         | 86             | 88        |   83                 |84
+| TechSlides                             | 90         | 93             | 92        |   91            |93
+| **Avg (REAL-MM-RAG)**                  | **73**     | **81**         | **85**    |   **74**           |**80**
 - **Release Date**: June 11th 2025
 - **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
 For an example of MM-RAG using granite-vision-3.3-2b-embedding refer to [this notebook](......).
 **Model Architecture:**
+The architecture of granite-vision-3.3-2b-embedding follows ColPali(https://arxiv.org/abs/2407.01449) approach and consists of the following components:
+(1) Vision-Language model : granite-vision-3.3-2b (https://huggingface.co/ibm-granite/granite-vision-3.3-2b).
+(2) Projection layer: linear layer that projects the hidden layer dimension of Vision-Language model to 128 and outputs 729 embedding vectors per image.
+The scoring is computed using MaxSim-based late interaction mechanism.
 **Training Data:**
 Our training data is entirly comprised from DocFM. DocFM is a large-scale comprehensive dataset effort at IBM consisting of 85 million document pages extracted from unique PDF