ychu612/BioClinicalBERT_rsavav_fn_adult2

README.md

@@ -33,14 +33,17 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 1.0089271157862022e-05
-- train_batch_size: 8
+- learning_rate: 2.2e-05
+- train_batch_size: 16
 - eval_batch_size: 8
 - seed: 42
+- gradient_accumulation_steps: 3
+- total_train_batch_size: 48
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_steps: 500
-- num_epochs: 3
+- num_epochs: 5
+- mixed_precision_training: Native AMP
 
 ### Training results
 
@@ -48,7 +51,7 @@ The following hyperparameters were used during training:
 
 ### Framework versions
 
-- Transformers 4.45.2
-- Pytorch 2.2.1
-- Datasets 3.0.1
-- Tokenizers 0.20.0
+- Transformers 4.44.2
+- Pytorch 2.4.0+cu121
+- Datasets 2.21.0
+- Tokenizers 0.19.1

config.json

@@ -59,7 +59,7 @@
   "position_embedding_type": "absolute",
   "problem_type": "single_label_classification",
   "torch_dtype": "float32",
-  "transformers_version": "4.45.2",
+  "transformers_version": "4.44.2",
   "type_vocab_size": 2,
   "use_cache": true,
   "vocab_size": 28996

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a1562862ef7b609e69d5b5770b4160756683bc87d8d963ba01f8ad03f9cc5f41
+oid sha256:cdfa59640ba08da07f12f0ff6dd11d6c2bae9e6bfaca0e52fa3abb0c04807e00
 size 433319984

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1fec23ec457f1b0ccdd211d0e3c8fdb7e56b6999a2c8af42e68976a0d002e156
-size 5304
+oid sha256:d9ce027f9c8658425e047b12b27d5951029b4d05265d6d3867a8961be8c50f87
+size 5240