title: validation loss higher only for some tasks
body: <p>I am training a multi task network, it seems that the validation loss is higher than the training loss only for some tasks but for others, the network seems to converge pretty well. For one task in particular, the validation loss is much higher than the training one and it affects the average. I added some data augmentation, normalization, dropouts, batch norm etc. to avoid overfitting in general. How can I handle this one single task??? </p>