MTLClassifier.load_and_evaluate_test_model error
Dear,
I am trying to fine tune my model, But I get error due to change in state dict.
RuntimeError: Error(s) in loading state_dict for GeneformerMultiTask:
size mismatch for classification_heads.0.weight: copying a param with shape torch.Size([64, 512]) from checkpoint, the shape in current model is torch.Size([62, 512]).
size mismatch for classification_heads.0.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([62]).
Hi there! Could you provide the traceback if possible and also whether you are running the the fine-tuned model on the validation or test set?
Thanks,
Madhavan
Hello,
mc.load_and_evaluate_test_model()
/usr/local/lib/python3.11/site-packages/torch/utils/data/dataloader.py:560: UserWarning: This DataLoader will create 32 worker processes in total. Ou
r suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
Loaded hyperparameters:
learning_rate: 1.0675913490772817e-05
warmup_ratio: 0.006323199961073651
weight_decay: 0.08177185852512817
dropout_rate: 0.6397931101620855
lr_scheduler_type: cosine
use_attention_pooling: False
task_weight_0: 0.10134751409827675
task_weight_1: 1.1994109758258542
max_layers_to_freeze: 0
normalized_task_weights: [0.07791416691363516, 0.9220858330863649]
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.11/site-packages/geneformer/mtl_classifier.py", line 347, in load_and_evaluate_test_model
eval_utils.load_and_evaluate_test_model(self.config)
File "/usr/local/lib/python3.11/site-packages/geneformer/mtl/eval_utils.py", line 84, in load_and_evaluate_test_model
best_model.load_state_dict(torch.load(best_model_path))
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for GeneformerMultiTask:
size mismatch for classification_heads.0.weight: copying a param with shape torch.Size([64, 512]) from checkpoint, the shape in current model is torch.Size([62, 512]).
size mismatch for classification_heads.0.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([62]).
Im running them on the test set. mc.load_and_evaluate_test_model() "
Thank you