Push model using huggingface_hub.

README.md

@@ -1,147 +1,9 @@
 ---
-base_model: google-bert/bert-base-uncased
-datasets:
-- prithivMLmods/Spam-Text-Detect-Analysis
-license: apache-2.0
 tags:
-- embedding_space_map
-- BaseLM:google-bert/bert-base-uncased
+- model_hub_mixin
+- pytorch_model_hub_mixin
 ---
 
-# ESM prithivMLmods/Spam-Text-Detect-Analysis
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-ESM
-
-- **Developed by:** [Unknown]
-- **Model type:** ESM
-- **Base Model:** google-bert/bert-base-uncased
-- **Intermediate Task:** prithivMLmods/Spam-Text-Detect-Analysis
-- **ESM architecture:** linear
-- **ESM embedding dimension:** 768
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** Apache-2.0 license
-
-## Training Details
-
-### Intermediate Task
-- **Task ID:** prithivMLmods/Spam-Text-Detect-Analysis
-- **Subset [optional]:** 
-- **Text Column:** Message
-- **Label Column:** Category
-- **Dataset Split:**  train
-- **Sample size [optional]:** 1000
-- **Sample seed [optional]:** 
-
-### Training Procedure [optional]
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Language Model Training Hyperparameters [optional]
-- **Epochs:** [More Information Needed]
-- **Batch size:** [More Information Needed]
-- **Learning rate:** [More Information Needed]
-- **Weight Decay:** [More Information Needed]
-- **Optimizer**: [More Information Needed]
-
-### ESM Training Hyperparameters [optional]
-- **Epochs:** 10
-- **Batch size:** 32
-- **Learning rate:** 0.001
-- **Weight Decay:** 0.01
-- **Optimizer**: [More Information Needed]
-
-
-### Additional trainiung details [optional]
-
-
-## Model evaluation
-
-### Evaluation of fine-tuned language model [optional]
-
-
-### Evaluation of ESM [optional]
-MSE: 
-
-### Additional evaluation details [optional]
-
-
-
-## What are Embedding Space Maps?
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-Embedding Space Maps (ESMs) are neural networks that approximate the effect of fine-tuning a language model on a task. They can be used to quickly transform embeddings from a base model to approximate how a fine-tuned model would embed the the input text.
-ESMs can be used for intermediate task selection with the ESM-LogME workflow.
-
-## How can I use Embedding Space Maps for Intermediate Task Selection?
-[![PyPI version](https://img.shields.io/pypi/v/hf-dataset-selector.svg)](https://pypi.org/project/hf-dataset-selector)
-
-We release **hf-dataset-selector**, a Python package for intermediate task selection using Embedding Space Maps.
-
-**hf-dataset-selector** fetches ESMs for a given language model and uses it to find the best dataset for applying intermediate training to the target task. ESMs are found by their tags on the Huggingface Hub.
-
-```python
-from hfselect import Dataset, compute_task_ranking
-
-# Load target dataset from the Hugging Face Hub
-dataset = Dataset.from_hugging_face(
-    name="stanfordnlp/imdb",
-    split="train",
-    text_col="text",
-    label_col="label",
-    is_regression=False,
-    num_examples=1000,
-    seed=42
-)
-
-# Fetch ESMs and rank tasks
-task_ranking = compute_task_ranking(
-    dataset=dataset,
-    model_name="bert-base-multilingual-uncased"
-)
-
-# Display top 5 recommendations
-print(task_ranking[:5])
-```
-
-For more information on how to use ESMs please have a look at the [official Github repository](https://github.com/davidschulte/hf-dataset-selector).
-
-## Citation
-
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-If you are using this Embedding Space Maps, please cite our [paper](https://arxiv.org/abs/2410.15148).
-
-**BibTeX:**
-
-
-```
-@misc{schulte2024moreparameterefficientselectionintermediate,
-      title={Less is More: Parameter-Efficient Selection of Intermediate Tasks for Transfer Learning}, 
-      author={David Schulte and Felix Hamborg and Alan Akbik},
-      year={2024},
-      eprint={2410.15148},
-      archivePrefix={arXiv},
-      primaryClass={cs.CL},
-      url={https://arxiv.org/abs/2410.15148}, 
-}
-```
-
-
-**APA:**
-
-```
-Schulte, D., Hamborg, F., & Akbik, A. (2024). Less is More: Parameter-Efficient Selection of Intermediate Tasks for Transfer Learning. arXiv preprint arXiv:2410.15148.
-```
-
-## Additional Information
-
+This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
+- Library: [More Information Needed]
+- Docs: [More Information Needed]

config.json

@@ -1,27 +1,89 @@
 {
+  "_attn_implementation_autoset": false,
+  "_name_or_path": "",
+  "add_cross_attention": false,
+  "architectures": null,
+  "bad_words_ids": null,
   "base_model_name": "google-bert/bert-base-uncased",
+  "begin_suppress_tokens": null,
+  "bos_token_id": null,
+  "chunk_size_feed_forward": 0,
+  "cross_attention_hidden_size": null,
+  "decoder_start_token_id": null,
   "developers": null,
+  "diversity_penalty": 0.0,
+  "do_sample": false,
+  "early_stopping": false,
+  "encoder_no_repeat_ngram_size": 0,
+  "eos_token_id": null,
   "esm_architecture": "linear",
   "esm_batch_size": 32,
   "esm_embedding_dim": 768,
-  "esm_learning_rate": 0.001,
+  "esm_learning_rate": 0.01,
   "esm_num_epochs": 10,
   "esm_optimizer": null,
   "esm_weight_decay": 0.01,
+  "exponential_decay_length_penalty": null,
+  "finetuning_task": null,
+  "forced_bos_token_id": null,
+  "forced_eos_token_id": null,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1"
+  },
+  "is_decoder": false,
+  "is_encoder_decoder": false,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1
+  },
   "label_column": "Category",
   "language": null,
+  "length_penalty": 1.0,
   "lm_batch_size": null,
   "lm_learning_rate": null,
   "lm_num_epochs": null,
   "lm_optimizer": null,
   "lm_weight_decay": null,
+  "max_length": 20,
+  "min_length": 0,
+  "model_type": "",
+  "no_repeat_ngram_size": 0,
+  "num_beam_groups": 1,
+  "num_beams": 1,
   "num_examples": 1000,
+  "num_return_sequences": 1,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "output_scores": false,
+  "pad_token_id": null,
+  "prefix": null,
+  "problem_type": null,
+  "pruned_heads": {},
+  "remove_invalid_values": false,
+  "repetition_penalty": 1.0,
+  "return_dict": true,
+  "return_dict_in_generate": false,
   "seed": null,
+  "sep_token_id": null,
   "streamed": false,
+  "suppress_tokens": null,
   "task_id": "prithivMLmods/Spam-Text-Detect-Analysis",
+  "task_specific_params": null,
   "task_split": "train",
   "task_subset": null,
+  "temperature": 1.0,
   "text_column": "Message",
+  "tf_legacy_loss": false,
+  "tie_encoder_decoder": false,
+  "tie_word_embeddings": true,
+  "tokenizer_class": null,
+  "top_k": 50,
+  "top_p": 1.0,
+  "torch_dtype": null,
+  "torchscript": false,
   "transformers_version": "4.47.1",
+  "typical_p": 1.0,
+  "use_bfloat16": false,
   "version": "0.2.0"
-}
+}

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fcd6e6aebba89435b42ee3abd0c3d2030bd07df55b32738deaa0ec81f29775dd
+oid sha256:2c341a7ec672afe7740ce0321d8e7ed2901050c6e2cf658eaac9941e176aedd4
 size 2362528